Hello everyone
I have a question for the following code. It's a reduced variant of the actual code that still replicates the problem.
I am parallelizing a task over a list of particles. The algorithm I use the calculate (get_chunk_size in the code below) can end up producing zero particles for the last thread. Here for example I have 5 particles and 4 threads which ends up distributing the particles like [2, 2, 1, 0].
A different algorithm would rather do [2, 1, 1, 1].
The reason I use the first algorithm is because it is super simple and easy to convert from an index in the particle list to an index in the chunked particle list. Also for the runtime the simulation with typical particle numbers of 1024*1024*1024 it doesn't really matter if the last thread is empty and all other threads have a few more particles.
So the problem I have now is the following. How do I let the last thread just handle 0 particles? I think if I could just produce an empty slice it would be fine. I am trying to do this with zip_longest but I can't produce an empty slice then, because it's lifetime would have to be longer than thread::scope() s lifetime since particles also has this longer lifetime.
Do you have an idea how I could solve this? I am also open to suggestions that just circumvent the problem.
use itertools::Itertools;
use std::thread;
fn main() {
const MIN: f32 = -0.5;
const MAX: f32 = 0.5;
let mut particles = vec![
[MIN, MIN],
[MIN, MIN],
[MAX, MAX],
[MIN, (MAX + MIN) / 2.],
[(MAX + MIN) / 2., (MAX + MIN) / 2.],
];
let n_particles = particles.len();
const N_GRID: usize = 4;
const N_THREADS: usize = 4;
// Number of particles per thread.
let chunk_size: usize = get_chunk_size(n_particles, N_THREADS);
let mut communicators = vec![0; N_THREADS];
thread::scope(|s| {
for either_or_both in communicators
.iter_mut()
.zip_longest(particles.chunks_mut(chunk_size))
{
let mut v = vec![];
let (_comm, p_local) = match either_or_both {
itertools::EitherOrBoth::Both(comm, p_local) => (comm, p_local),
itertools::EitherOrBoth::Left(comm) => (comm, &mut v[..]),
itertools::EitherOrBoth::Right(_) => panic!("This shouldn't happen."),
};
s.spawn(|| {
// Do something with p_local.
p_local;
});
}
});
}
fn get_chunk_size(len: usize, n_chunks: usize) -> usize {
(len + n_chunks - 1) / n_chunks
}
Errors:
Compiling playground v0.0.1 (/playground)
error[E0597]: `v` does not live long enough
--> src/main.rs:32:68
|
24 | thread::scope(|s| {
| - has type `&'1 Scope<'1, '_>`
...
32 | itertools::EitherOrBoth::Left(comm) => (comm, &mut v[..]),
| ^ borrowed value does not live long enough
...
36 | / s.spawn(|| {
37 | | // Do something with p_local.
38 | | p_local;
39 | | });
| |______________- argument requires that `v` is borrowed for `'1`
40 | }
| - `v` dropped here while still borrowed
For more information about this error, try `rustc --explain E0597`.
error: could not compile `playground` due to previous error
