Cannot borrow as mutable, as it is a captured variable in a `Fn` closure

1

There are two ndarrays, a and b
I would like to parallel iterate through array a, and update array b at the same time.

a.par_iter().enumerate().for_each(|(i, data)| { b[i] = data + 1; });

However, I'm getting the message
"cannot borrow b as mutable, as it is a captured variable in a Fn closure"

I know that I could use par_iter_mut() instead, but this only enables mutable on a, but not b.

2

Try something like this (untested, on mobile):

a.par_iter().zip(b.par_iter_mut()).for_each(|(a_i, mut b_i)| { *b_i = a_i + 1; });
2 Likes
3

In case it isn't clear what the original message was about, maybe this section of a talk from the other day can give a glimpse of more insight. The error message can indeed be a bit inaccessible.

TL;DR, this limitation of mutability when mapping parallel iterators prevents data races.

In your use case, there wasn't actually a data race but that's only because of your special use-pattern where each iteration was always making sure to access a different, unique entry of the captured b.

Using zip as suggested above is the best solution here, because it's specifically marking mutable access in your access pattern as safe without further overhead from use of synchronization primitives.

(A more "naive" alternative approaches to make the code work like wrapping b with Mutex, might kill much of the parallelness of the code here.)

2 Likes
4

Using zip did the trick!
Thanks!

5

The video is quite useful!
Thanks!

As a follow-up question, how do I zip multiple arrays?
I'm currently using
a.par_iter().zip(b.par_iter_mut()).zip(c.par_iter_mut()).for_each(|((a_i, mut b_i), mut c_i) | { *b_i = a_i + c_i + 1; });
I notice that if I use multiple zips, then it will become
((a_i, b_i), c_i)
How can I do it like the following
(a_i, b_i, c_i)

6

The approach of nested tuples isn't bad. Still, there's alternatives. It looks like rayon offers an alternative API here: MultiZip in rayon::iter - Rust

You should be able to use that by just writing (&a, &mut b, &c).into_par_iter().

1 Like
7

For supporting ndarray in particular better, e. g. checking that the shapes actually properly match, you could also look into API that ndarray itself offers. I'm seeing this macro in particular: https://docs.rs/ndarray/latest/ndarray/macro.par_azip.html

1 Like
8

into_par_iter() works perfectly well in my case!

9

I couldn't use par_azip as the 2nd dimension of the arrays is different.
A is in dimension [n, m],
B is in dimension [n, p],
And I would like to iterate through the 1st dimension in parallel.
If I use par_azip, it will compile, however it
Panics at runtime because one of the arrays is not of the same shape.

10

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.