Rayon flat_map not matching regular flat_map


#1

I’m trying to convert a list of values into a list of values that will be twice the size via a look-up table. I want to use Rayon to speed things up a bit. I have a simple reproducer Playground:

    let lookup :Vec<[i32; 2]> = Vec::with_capacity(10);
    
    for i in 0..10 {
        lookup.push([-1*i, i]);
    }

    let in_vals = (0..10).collect::<Vec<_>>();
    let mut out_vals = Vec::<i32>::with_capacity(in_vals.len()*2);
    
    let p_iter = in_vals.par_iter().flat_map(|v| {*lookup.get(*v as usize).unwrap()});
    
    out_vals.par_extend(p_iter);

I keep getting this error:

error[E0277]: the trait bound `[i32; 2]: rayon::iter::ParallelIterator` is not satisfied
  --> src/main.rs:14:37
   |
14 |     let p_iter = in_vals.par_iter().flat_map(|v| {*lookup.get(*v as usize).unwrap()});
   |                                     ^^^^^^^^ the trait `rayon::iter::ParallelIterator` is not implemented for `[i32; 2]`
   |
   = note: required because of the requirements on the impl of `rayon::iter::IntoParallelIterator` for `[i32; 2]`

error[E0277]: the trait bound `[i32; 2]: rayon::iter::ParallelIterator` is not satisfied
  --> src/main.rs:16:14
   |
16 |     out_vals.par_extend(p_iter);
   |              ^^^^^^^^^^ the trait `rayon::iter::ParallelIterator` is not implemented for `[i32; 2]`
   |
   = note: required because of the requirements on the impl of `rayon::iter::IntoParallelIterator` for `[i32; 2]`
   = note: required because of the requirements on the impl of `rayon::iter::ParallelIterator` for `rayon::iter::flat_map::FlatMap<rayon::slice::Iter<'_, {integer}>, [closure@src/main.rs:14:46: 14:85 lookup:_]>`

When I remove the Rayon code, everything works as expected:

    let iter = in_vals.iter().flat_map(|v| {lookup.get(*v as usize).unwrap()});
    out_vals.extend(iter);

Can this simply not be done with Rayon? Thanks!


#2

As the error describes, [i32; 2] is not a ParallelIterator, so the correct version of this code is this:

let mut lookup: Vec<[i32; 2]> = Vec::with_capacity(10);
    
for i in 0..10 {
    lookup.push([-1*i, i]);
}

let in_vals = (0..10).collect::<Vec<_>>();
let mut out_vals = Vec::<i32>::with_capacity(in_vals.len()*2);
    
let p_iter = in_vals.par_iter().flat_map(|v| {lookup.get(*v as usize).unwrap().par_iter()});
out_vals.par_extend(p_iter);

Note the additional .par_iter() on the .unwrap()


#3

@OptimisticPeach, ah makes sense now that I see it… was slightly confused exactly where the [i32; 2] was coming from. Thanks!


#4

Another difference is that your parallel version dereferenced to a bare array value, whereas your serial version left it as a reference. The standard library implements IntoIterator for &[T; N] as if it were a slice. We could do the same for IntoParallelIterator, so it would implicitly work the same as the manual par_iter() that @OptimisticPeach used.