Rayon and mutexes

1

Hey there!

How does Rust figures out atomic reference counting is not required here?

extern crate rayon;

use rayon::prelude::*;
use std::sync::{Mutex};

fn main() {
    let vec: Mutex<Vec<usize>> = Mutex::new(vec![]);

    (0..100).collect::<Vec<usize>>().into_par_iter().for_each(|x| {
        let mut locked = vec.lock().unwrap();
        locked.push(x);
    });
}

Since Rayon blocks until the parallel iteration completes, I understand that there is no need to keep track of a reference count. However, I'm not sure to understand how Rust figures this out.

I have a hunch that it's because for_each takes a closure of type F: Fn(T) + Sync while thread::spawn takes a closure of type F: FnOnce() -> T, F: Send + 'static, T: Send + 'static.

Am I right? Could someone explain this to me?

2

Yes, it's the missing 'static requirement that allows the closure to borrow local (or more generally, current stack's) variables. Rayon has unsafe code internally to maintain the various invariants. If you were to mess this up, and the current thread proceeds, the locals would die and the other threads would dereference free'd memory.

There used to be a way to do this using std's thread functionality, which relied on a RAII guard to block waiting for the spawned thread to exit. However, this was unsound because of std::mem::forget'ing a value, and bypassing its Drop impl.

The crossbeam crate allows this functionality, but uses scopes from which you run a closure - there's no handle to std::mem::forget and thus that hazard is avoided.

3 Likes