I have a large vector of some datatype. I need to perform some computation on each entry, altering the entry each time. Therefore, the input is a mutable slice of elements. To speed up the computation, I have parallelised it. Specifically, each thread takes a chunk of elements from the "queue" and then performs the computation on each element. The chunk size is determined dynamically.
Now the question is, how can a thread take a chunk of elements out of the input slice. Consider the following code (which does not compile):
extern crate crossbeam;
use std::sync::Mutex;
fn main() {
let mut data = vec![0; 1024];
let shared_data = Mutex::new(data.as_mut_slice()); // a large amount of data
crossbeam::scope(|scope| {
for _ in 0..4 {
scope.spawn(|_| {
loop {
let chunk_size = 10; // dynamically computed, different each iteration
let local_data: &mut [i32] = {
let mut locked_shared_data = shared_data.lock().unwrap();
let (local_data, remaining_shared_data) =
locked_shared_data.split_at_mut(chunk_size);
*locked_shared_data = remaining_shared_data;
local_data
};
// perform computations, reading from and writing to local_data
drop(local_data);
}
});
}
}).unwrap();
}
Errors
Compiling playground v0.0.1 (/playground)
error[E0597]: `locked_shared_data` does not live long enough
--> src/main.rs:18:33
|
6 | let shared_data = Mutex::new(data.as_mut_slice()); // a large amount of data
| ----------- lifetime `'1` appears in the type of `shared_data`
...
18 | locked_shared_data.split_at_mut(chunk_size);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| |
| borrowed value does not live long enough
| argument requires that `locked_shared_data` is borrowed for `'1`
...
22 | };
| - `locked_shared_data` dropped here while still borrowed
error[E0499]: cannot borrow `locked_shared_data` as mutable more than once at a time
--> src/main.rs:19:26
|
18 | locked_shared_data.split_at_mut(chunk_size);
| ------------------------------------------- first mutable borrow occurs here
19 | *locked_shared_data = remaining_shared_data;
| ^^^^^^^^^^^^^^^^^^ second mutable borrow occurs here
20 |
21 | local_data
| ---------- first borrow later used here
Some errors have detailed explanations: E0499, E0597.
For more information about an error, try `rustc --explain E0499`.
error: could not compile `playground` due to 2 previous errors
The input mutable slice is shared with a mutex between the threads. Whenever a thread needs more data, it locks the mutex and takes a subslice of the data. To avoid creating duplicate mutable references, it leaves only the remaining slice in the mutex. Example:
initial mutable slice inside the mutex:
shared_data == 0123456789
take a mutable subslice of length 3 out of the data:
shared_data == 3456789
local_data == 012
The thread can now work on local_data
, and no other thread can get mutable references to it, since shared_data
now points to a non-overlapping slice.
However, the compiler does not seem to be convinced by this. I get that the first error means that just because we overwrite the value inside the mutex, we cannot alter its associated lifetime. The second error is a bit confusing, since the "second mutable borrow" is actually overwriting the value inside the variable, as opposed to borrowing it?
Is there any way I can make this code compile in safe Rust? Or if not, in unsafe Rust?